<html>
<head><meta charset="utf-8"><title>-Zself-profile · t-compiler · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/index.html">t-compiler</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html">-Zself-profile</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="148882704"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/148882704" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#148882704">(Nov 30 2018 at 19:22)</a>:</h4>
<p>BTW <span class="user-mention" data-user-id="125250">@Wesley Wiser</span> are you currently working on extending <code>-Zself-profile</code> in any way?</p>
<p>I ran against the style-servo crate and I got this:</p>
<div class="codehilite"><pre><span></span>| Phase            | Time (ms)      | Queries        | Hits (%) |
| ---------------- | -------------- | -------------- | -------- |
| Parsing          | 143            |                |          |
| Expansion        | 3532           |                |          |
| TypeChecking     | 16340          | 95991164       | 97.21    |
| BorrowChecking   | 1525           | 718742         | 64.41    |
| Codegen          | 33423          | 3779295        | 87.41    |
| Linking          | 614            | 474496         | 91.10    |
| Other            | 7901           | 85816990       | 97.52    |
</pre></div>


<p>Seem like one obvious thing is it'd be nice to have %ages for the time in each of those categories</p>



<a name="148882731"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/148882731" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#148882731">(Nov 30 2018 at 19:23)</a>:</h4>
<p><span class="user-mention" data-user-id="116009">@nikomatsakis</span> <span class="user-mention" data-user-id="124287">@mw</span> asked for that the other day</p>



<a name="148882733"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/148882733" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#148882733">(Nov 30 2018 at 19:23)</a>:</h4>
<p>It's on my TODO list :)</p>



<a name="148882737"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/148882737" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#148882737">(Nov 30 2018 at 19:23)</a>:</h4>
<p>heh ok</p>



<a name="148882812"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/148882812" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#148882812">(Nov 30 2018 at 19:24)</a>:</h4>
<p>I'm in the middle of tracking down an issue with the inline changes I've been working on but after I've fixed this one, I'll take a break on work on the profiler</p>



<a name="148882853"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/148882853" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#148882853">(Nov 30 2018 at 19:25)</a>:</h4>
<p>ok</p>



<a name="152655637"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/152655637" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#152655637">(Dec 28 2018 at 14:15)</a>:</h4>
<p>So <span class="user-mention" data-user-id="125250">@Wesley Wiser</span> I think the next steps would be:</p>
<ul>
<li>Gather data per-query instead of per-category so that we can emit more detailed information</li>
<li>Display percentages and perhaps a bit more info -- but I suppose if we can emit JSON or CSV we can kind of play with that independently</li>
<li>Review the categories (probably other members of compiler team are better suited)</li>
</ul>
<p>Longer term:</p>
<ul>
<li>Optionally gather "query call-stack" information -- this would add overhead, but it seems like we should be able to do it very reliably, and account for the extra overhead. The idea would be to be able to answer questions like "what %age of the time was spent in query X when invoked from query Y" or "who invokes query X", that sort of thing.</li>
</ul>



<a name="152655640"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/152655640" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#152655640">(Dec 28 2018 at 14:16)</a>:</h4>
<p><span class="user-mention" data-user-id="124287">@mw</span> may have thoughts, though</p>



<a name="152655681"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/152655681" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#152655681">(Dec 28 2018 at 14:16)</a>:</h4>
<p>although I think they are on PTO :)</p>



<a name="152655905"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/152655905" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#152655905">(Dec 28 2018 at 14:23)</a>:</h4>
<blockquote>
<p>Display percentages and perhaps a bit more info -- but I suppose if we can emit JSON or CSV we can kind of play with that independently</p>
</blockquote>
<p>Do you mean like this? <a href="https://github.com/rust-lang/rust/pull/56702" target="_blank" title="https://github.com/rust-lang/rust/pull/56702">https://github.com/rust-lang/rust/pull/56702</a></p>



<a name="152656411"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/152656411" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#152656411">(Dec 28 2018 at 14:37)</a>:</h4>
<p>I suppose I do =)</p>



<a name="152656418"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/152656418" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#152656418">(Dec 28 2018 at 14:37)</a>:</h4>
<p><span class="user-mention" data-user-id="125250">@Wesley Wiser</span> do we have JSON or CSV output?</p>



<a name="152656462"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/152656462" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#152656462">(Dec 28 2018 at 14:38)</a>:</h4>
<p>JSON</p>



<a name="152656472"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/152656472" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#152656472">(Dec 28 2018 at 14:38)</a>:</h4>
<p>Thanks to <span class="user-mention" data-user-id="116122">@simulacrum</span>, it's actually being collected on perf.rlo <a href="https://github.com/rust-lang-nursery/rustc-perf/issues/299#issuecomment-444736451" target="_blank" title="https://github.com/rust-lang-nursery/rustc-perf/issues/299#issuecomment-444736451">https://github.com/rust-lang-nursery/rustc-perf/issues/299#issuecomment-444736451</a></p>



<a name="152656481"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/152656481" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#152656481">(Dec 28 2018 at 14:39)</a>:</h4>
<p>There's no visualization yet though</p>



<a name="152656487"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/152656487" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#152656487">(Dec 28 2018 at 14:39)</a>:</h4>
<p>Yep, we're waiting on a proposal though mainly. FWIW I have the time to invest in the next ~week or two so proposals are welcome :)</p>



<a name="155250658"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155250658" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155250658">(Jan 16 2019 at 12:50)</a>:</h4>
<p>I'd like to re-evaluate what we want from -Zself-profile and then set some goals from there.</p>



<a name="155250796"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155250796" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155250796">(Jan 16 2019 at 12:52)</a>:</h4>
<p>Ideally self-profile would answered all questions about "where is the compiler spending its time". perf and cachegrind are not so good for that (for various reasons).</p>



<a name="155250981"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155250981" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155250981">(Jan 16 2019 at 12:55)</a>:</h4>
<p>So some of the requirements for this are:</p>
<ul>
<li>Measure the total time the compiler process runs. This seems trivial but -Ztime-passes for example never did it. This seems to be a prerequisite for being able to tell whether we are measuring everything.</li>
</ul>



<a name="155251186"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155251186" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Björn Steinbrink <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155251186">(Jan 16 2019 at 12:58)</a>:</h4>
<p>A better break-down of which queries are actually executed would be nice. Just seeing <code>Other</code> or <code>Type checking</code> taking 80% of the time isn't all that useful.</p>



<a name="155251192"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155251192" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155251192">(Jan 16 2019 at 12:59)</a>:</h4>
<ul>
<li>Record individual query invocations. It seems like for a real profile we want to be able know which query ran for how long on what thread. From this raw data, we can build tools for rich visualization</li>
</ul>



<a name="155251220"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155251220" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155251220">(Jan 16 2019 at 12:59)</a>:</h4>
<blockquote>
<p>A better break-down of which queries are actually executed would be nice.</p>
</blockquote>
<p>Agreed, but I think want to go even a step further and record actual start and end timestamps.</p>



<a name="155251235"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155251235" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Björn Steinbrink <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155251235">(Jan 16 2019 at 12:59)</a>:</h4>
<p>I also noticed that the time attribution seems to be a bit off. Time increased in one area, but the query count in another. Most likely this is because the timing only starts if the query is not answered from cache?</p>



<a name="155251368"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155251368" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Björn Steinbrink <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155251368">(Jan 16 2019 at 13:01)</a>:</h4>
<p>How much more info does -Zprofile-queries already provide? The only time I tried to use that, rustc got OOM'd, 32GB just wasn't enough</p>



<a name="155251387"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155251387" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155251387">(Jan 16 2019 at 13:02)</a>:</h4>
<ul>
<li>Record more data about query execution.  Cache-hit, cache-miss, found-in-incremental-cache, would-be-reusable-but-we-dont-cache-it, etc. I.e. make -Zincremental-info obsolete</li>
</ul>



<a name="155251520"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155251520" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155251520">(Jan 16 2019 at 13:03)</a>:</h4>
<blockquote>
<p>How much more info does -Zprofile-queries already provide? </p>
</blockquote>
<p>I think it collects a lot of data but is not optimized enough for actual profiling.</p>



<a name="155251610"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155251610" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155251610">(Jan 16 2019 at 13:04)</a>:</h4>
<p>I probably stores query keys too.</p>



<a name="155251655"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155251655" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155251655">(Jan 16 2019 at 13:05)</a>:</h4>
<p>but to be honest, I've never used it in earnest and would opt for removing it from the compiler</p>



<a name="155251754"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155251754" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155251754">(Jan 16 2019 at 13:07)</a>:</h4>
<ul>
<li>self-profile should also record instances of lock-contention during query invocation.</li>
</ul>



<a name="155251934"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155251934" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155251934">(Jan 16 2019 at 13:09)</a>:</h4>
<ul>
<li>we need to keep supporting profiling outside of the query system too. At least in the medium term.</li>
</ul>



<a name="155252192"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155252192" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155252192">(Jan 16 2019 at 13:13)</a>:</h4>
<ul>
<li>we need to support at least two levels of profiling detail. Collecting the full profiles that I suggested above is probably too much for perf.rlo. Although the broad categorization we have at the moment is not so useful yet either. I suggest that  we keep the categories but also collect accumulated times per-query when doing a non-detailed profile. <br>
EDIT: If there is a cheap way to also record the net time spent in a query (i.e. total time minus time spent in sub-queries) that would be nice to have too. The detailed profiling data has the information via timestamps, but maybe we can also do it for summary queries somehow.</li>
</ul>



<a name="155252769"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155252769" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155252769">(Jan 16 2019 at 13:22)</a>:</h4>
<p>I think with detailed profiles like described above we can finally easily answer questions like:</p>
<ul>
<li>Metadata encoding (or what have you) seems to take a really long time. What portion of that is spent in queries being invoked the first time, and what is the actual cost of it?</li>
<li>How much do parallel queries suffer from contention and where?</li>
<li>The times don't add up to the total time, what are we missing?</li>
</ul>



<a name="155252947"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155252947" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155252947">(Jan 16 2019 at 13:25)</a>:</h4>
<ul>
<li>How much time do we spend in trait selection, and which queries account for how much of that?</li>
</ul>



<a name="155253229"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155253229" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155253229">(Jan 16 2019 at 13:30)</a>:</h4>
<p>Can anyone think of anything else? Does anyone see a problem with my proposed requirements?</p>



<a name="155256557"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155256557" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155256557">(Jan 16 2019 at 14:21)</a>:</h4>
<p>It would be nice to have a way to see the query stacks per query. IE: <code>query a</code> was invoked from <code>query b</code> which was invoked from <code>query c</code>.</p>



<a name="155256655"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155256655" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155256655">(Jan 16 2019 at 14:22)</a>:</h4>
<p>yes, that would be cool</p>



<a name="155256679"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155256679" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155256679">(Jan 16 2019 at 14:22)</a>:</h4>
<p>I mean, this can be inferred once you have time-stamps</p>



<a name="155256702"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155256702" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155256702">(Jan 16 2019 at 14:23)</a>:</h4>
<p>Probably only in single threaded mode right?</p>



<a name="155256716"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155256716" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155256716">(Jan 16 2019 at 14:23)</a>:</h4>
<p>I'd like to record time stamps per thread</p>



<a name="155256735"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155256735" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155256735">(Jan 16 2019 at 14:23)</a>:</h4>
<p>Oh ok</p>



<a name="155256801"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155256801" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155256801">(Jan 16 2019 at 14:24)</a>:</h4>
<p>I was thinking that we can have a "profile" mode that records basically everything about execution as a series of "per thread events", right? I thnk this is probably what <span class="user-mention" data-user-id="124287">@mw</span> is describing too. Then we can reconstruct whatever view we want from that</p>



<a name="155256859"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155256859" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155256859">(Jan 16 2019 at 14:24)</a>:</h4>
<p>yes</p>



<a name="155256881"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155256881" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155256881">(Jan 16 2019 at 14:25)</a>:</h4>
<p>it should not be invasive performance-wise</p>



<a name="155256882"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155256882" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155256882">(Jan 16 2019 at 14:25)</a>:</h4>
<p>e.g., you might log:</p>
<ul>
<li>Thread T1 started Executing Query Q1 at timestamp N1</li>
<li>Thread T1 started Executing Query Q2 at timestamp N2 </li>
<li>Thread T1 finished Executing Query Q2 at timestamp N2</li>
<li>Thread T1 had cache hit for Query Q2<br>
...</li>
</ul>



<a name="155256893"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155256893" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155256893">(Jan 16 2019 at 14:25)</a>:</h4>
<p>I feel like..even if it is... we can kind of "account for it"</p>



<a name="155256902"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155256902" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155256902">(Jan 16 2019 at 14:25)</a>:</h4>
<p>but it probably shouldn't that expensive</p>



<a name="155256905"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155256905" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155256905">(Jan 16 2019 at 14:25)</a>:</h4>
<p>but if we only record the data but don't process it immediately that should be fine</p>



<a name="155256912"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155256912" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155256912">(Jan 16 2019 at 14:25)</a>:</h4>
<p>yes, that</p>



<a name="155256928"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155256928" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155256928">(Jan 16 2019 at 14:25)</a>:</h4>
<p>right now at least our compilation times are sufficiently big that I am not very worried about this at all =)</p>



<a name="155256983"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155256983" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155256983">(Jan 16 2019 at 14:26)</a>:</h4>
<p>:)</p>



<a name="155257051"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155257051" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155257051">(Jan 16 2019 at 14:26)</a>:</h4>
<blockquote>
<p>EDIT: If there is a cheap way to also record the net time spent in a query (i.e. total time minus time spent in sub-queries) that would be nice to have too. The detailed profiling data has the information via timestamps, but maybe we can also do it for summary queries somehow.</p>
</blockquote>
<p>This is actually what the summary times currently show. We need to start tracking at the query level though instead of just the category level to get most of the other requirements.</p>



<a name="155257199"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155257199" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155257199">(Jan 16 2019 at 14:28)</a>:</h4>
<p>so if a "TypeCheck" query calls an "Other" query, the execution time of the "Other" query is substracted from the "TypeCheck" query?</p>



<a name="155257378"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155257378" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155257378">(Jan 16 2019 at 14:30)</a>:</h4>
<p>Yes, exactly</p>



<a name="155257584"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155257584" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155257584">(Jan 16 2019 at 14:33)</a>:</h4>
<p>it seems like producing the log is step 1</p>



<a name="155257752"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155257752" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155257752">(Jan 16 2019 at 14:35)</a>:</h4>
<p>It would be great if this also worked out of the box with parallel queries.</p>



<a name="155257773"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155257773" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155257773">(Jan 16 2019 at 14:35)</a>:</h4>
<p>I don't really have a handle on how parallel queries works though</p>



<a name="155258981"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155258981" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155258981">(Jan 16 2019 at 14:50)</a>:</h4>
<p>I don't think that's a big problem</p>



<a name="155258986"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/155258986" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#155258986">(Jan 16 2019 at 14:50)</a>:</h4>
<p>as long as we make the logs "thread-local" basically</p>



<a name="157144875"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157144875" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157144875">(Jan 29 2019 at 22:34)</a>:</h4>
<p>I have an initial (incomplete) implementation which gives <a href="https://gist.github.com/wesleywiser/e86e5212da50947f43274cb5e2137208" target="_blank" title="https://gist.github.com/wesleywiser/e86e5212da50947f43274cb5e2137208">this output</a></p>



<a name="157144942"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157144942" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157144942">(Jan 29 2019 at 22:35)</a>:</h4>
<p>To make the output easier to understand, I decided to omit queries that took less than 1ms in total. Feel free to push back if you don't like that.</p>



<a name="157145005"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157145005" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157145005">(Jan 29 2019 at 22:36)</a>:</h4>
<p>Could you right align the time columns? =P</p>



<a name="157145057"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157145057" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157145057">(Jan 29 2019 at 22:37)</a>:</h4>
<p>Probably :)</p>



<a name="157177303"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157177303" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157177303">(Jan 30 2019 at 10:54)</a>:</h4>
<p>Nice!</p>



<a name="157177423"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157177423" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157177423">(Jan 30 2019 at 10:56)</a>:</h4>
<p>It would be great if there was a way to easily spot the most expensive queries. Maybe omit the category and just sort them?</p>



<a name="157177571"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157177571" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157177571">(Jan 30 2019 at 10:59)</a>:</h4>
<p>You know what would be awesome for the verbose profiling mode: If we store absolute timestamps (i.e. not relative to the beginning of the <code>rustc</code> process) we can profile the entire crate graph compilation, not just a single crate.</p>



<a name="157236662"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157236662" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157236662">(Jan 31 2019 at 00:45)</a>:</h4>
<p><span class="user-mention" data-user-id="124287">@mw</span>  They are sorted most-to-least expensive under the categories. I think I'll also sort the categories by total time so the most expensive query should be at the top.</p>
<p>We're also storing absolute times in the event log so that is possible :)</p>



<a name="157244069"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157244069" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157244069">(Jan 31 2019 at 03:27)</a>:</h4>
<div class="codehilite"><pre><span></span>Self profiling results:

| Phase                                     | Time (ms)      | Time (%) | Queries        | Hits (%)
| ----------------------------------------- | -------------- | -------- | -------------- | --------
| Other                                     |            417 |    54.02 |             -1 |       -1
| - {time spent not running queries}        |            352 |    45.60 |             -1 |       -1
| - const_eval                              |             16 |     2.07 |             -1 |       -1
| - type_of                                 |             10 |     1.30 |             -1 |       -1
| - adt_def                                 |              8 |     1.04 |             -1 |       -1
| - const_eval_raw                          |              8 |     1.04 |             -1 |       -1
| - visible_parent_map                      |              7 |     0.91 |             -1 |       -1
| - item_attrs                              |              6 |     0.78 |             -1 |       -1
| - item_children                           |              6 |     0.78 |             -1 |       -1
| - adt_dtorck_constraint                   |              3 |     0.39 |             -1 |       -1
| - adt_destructor                          |              1 |     0.13 |             -1 |       -1
| Linking                                   |            245 |    31.74 |             -1 |       -1
| - {time spent not running queries}        |            245 |    31.74 |             -1 |       -1
| TypeChecking                              |             62 |     8.03 |             -1 |       -1
| - trait_impls_of                          |             10 |     1.30 |             -1 |       -1
| - typeck_item_bodies                      |              9 |     1.17 |             -1 |       -1
| - typeck_tables_of                        |              9 |     1.17 |             -1 |       -1
| - {time spent not running queries}        |              9 |     1.17 |             -1 |       -1
| - evaluate_obligation                     |              7 |     0.91 |             -1 |       -1
| - const_is_rvalue_promotable_to_static    |              5 |     0.65 |             -1 |       -1
| - is_copy_raw                             |              5 |     0.65 |             -1 |       -1
| - rvalue_promotable_map                   |              5 |     0.65 |             -1 |       -1
| - dropck_outlives                         |              2 |     0.26 |             -1 |       -1
| - layout_raw                              |              1 |     0.13 |             -1 |       -1
| Codegen                                   |             30 |     3.89 |             -1 |       -1
| - {time spent not running queries}        |             17 |     2.20 |             -1 |       -1
| - collect_and_partition_mono_items        |             11 |     1.42 |             -1 |       -1
| - mir_const                               |              1 |     0.13 |             -1 |       -1
| - mir_validated                           |              1 |     0.13 |             -1 |       -1
| Expansion                                 |             17 |     2.20 |             -1 |       -1
| - {time spent not running queries}        |             17 |     2.20 |             -1 |       -1
| BorrowChecking                            |              1 |     0.13 |             -1 |       -1
| - borrowck                                |              1 |     0.13 |             -1 |       -1
| Parsing                                   |              0 |     0.00 |             -1 |       -1

Optimization level: No
Incremental: off
</pre></div>



<a name="157261891"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157261891" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157261891">(Jan 31 2019 at 10:31)</a>:</h4>
<p>Yeah, I think that's a good solution. There is no clear chronological order to categories anyway.</p>



<a name="157274984"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157274984" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157274984">(Jan 31 2019 at 14:26)</a>:</h4>
<p><span class="user-mention" data-user-id="125250">@Wesley Wiser</span> are you currently capturing stack trace information btw?</p>



<a name="157274989"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157274989" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157274989">(Jan 31 2019 at 14:26)</a>:</h4>
<p>(Just curious, this is awesome work regardless! <span aria-label="tada" class="emoji emoji-1f389" role="img" title="tada">:tada:</span>)</p>



<a name="157274998"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157274998" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157274998">(Jan 31 2019 at 14:26)</a>:</h4>
<p>I'm wondering if we can use this to gather some data on some benchmarks before all hands</p>



<a name="157275706"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157275706" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> blitzerr <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157275706">(Jan 31 2019 at 14:37)</a>:</h4>
<p>This is really awesome <span class="user-mention" data-user-id="125250">@Wesley Wiser</span></p>



<a name="157275716"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157275716" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157275716">(Jan 31 2019 at 14:37)</a>:</h4>
<p>Thanks!</p>



<a name="157275849"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157275849" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157275849">(Jan 31 2019 at 14:39)</a>:</h4>
<p><span class="user-mention" data-user-id="116009">@nikomatsakis</span> Not currently but I don't think it would be difficult to add that. I'm not sure how to aggregate and display the stack trace data though.</p>
<p>Do you just want them dumped to a file or something? Or do we want to do some kind of automated deeper analysis?</p>



<a name="157276091"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157276091" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> blitzerr <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157276091">(Jan 31 2019 at 14:42)</a>:</h4>
<p><span class="user-mention" data-user-id="125250">@Wesley Wiser</span> <br>
<code>Time not spent running queries</code>  what does that indicate ?</p>



<a name="157276167"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157276167" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157276167">(Jan 31 2019 at 14:43)</a>:</h4>
<p><span class="user-mention" data-user-id="128294">@blitzerr</span> So, for example, in Codegen, we have some queries which are tagged as being codegen related. But we also just a general timer that runs when we call into LLVM. The "Time not spent running queries" represents that time spent in LLVM which isn't covered by a query.</p>



<a name="157276214"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157276214" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> blitzerr <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157276214">(Jan 31 2019 at 14:44)</a>:</h4>
<p>Is there a PR I can look at ?</p>



<a name="157276241"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157276241" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> blitzerr <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157276241">(Jan 31 2019 at 14:44)</a>:</h4>
<p>Ahh! Makes sense. thank you</p>



<a name="157276296"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157276296" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157276296">(Jan 31 2019 at 14:45)</a>:</h4>
<p>Not yet. It's all changes I have locally that I haven't pushed yet because it regresses the cache hits/misses data and the json output.</p>



<a name="157276323"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157276323" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157276323">(Jan 31 2019 at 14:45)</a>:</h4>
<p>I'm hoping to have a PR up in the next day or two</p>



<a name="157276329"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157276329" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> blitzerr <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157276329">(Jan 31 2019 at 14:45)</a>:</h4>
<p>So this profiling is for query based compilation ? I think, at this point not all compiler stages are query based. So we don't have profiling information for that ?</p>



<a name="157276421"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157276421" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157276421">(Jan 31 2019 at 14:46)</a>:</h4>
<p>We start a timer at the beginning of compilation in the "Other" category. So we should have coverage of 99% of time that rustc runs. It just won't be categorized super helpfully.</p>



<a name="157276737"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157276737" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157276737">(Jan 31 2019 at 14:51)</a>:</h4>
<blockquote>
<p>I'm not sure how to aggregate and display the stack trace data though.</p>
</blockquote>
<p>I wouldn't try. I would want an option that dumps out all the data in some format for others to piece through.</p>



<a name="157276757"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157276757" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157276757">(Jan 31 2019 at 14:51)</a>:</h4>
<p>Basically, we can provide the "flat data" in a convenient way, but if you want to do more advanced queries, we can provide a tool for that</p>



<a name="157276783"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157276783" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157276783">(Jan 31 2019 at 14:51)</a>:</h4>
<p>e.g., I'd probably adapt something like <a href="https://github.com/nikomatsakis/perf-focus" target="_blank" title="https://github.com/nikomatsakis/perf-focus">perf-focus</a></p>



<a name="157276873"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157276873" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157276873">(Jan 31 2019 at 14:52)</a>:</h4>
<p>(Also, depending how huge that data is, we could maybe even publish it from perf, which would be <strong>amazing</strong> -- we might need to adopt some compression techniques, though)</p>



<a name="157277004"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157277004" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157277004">(Jan 31 2019 at 14:54)</a>:</h4>
<blockquote>
<p>(Also, depending how huge that data is, we could maybe even publish it from perf, which would be <strong>amazing</strong> -- we might need to adopt some compression techniques, though)</p>
</blockquote>
<p><code>perf</code> the Linux tool or <code>perf.r-l.o</code>?</p>



<a name="157277156"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157277156" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157277156">(Jan 31 2019 at 14:56)</a>:</h4>
<p>I meant publish it from <a href="http://perf.rust-lang.org/" target="_blank" title="http://perf.rust-lang.org/">http://perf.rust-lang.org/</a></p>



<a name="157277158"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157277158" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157277158">(Jan 31 2019 at 14:56)</a>:</h4>
<p>perf.rlo I'd guess</p>



<a name="157277192"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157277192" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157277192">(Jan 31 2019 at 14:57)</a>:</h4>
<p>we'd have to make sure though that profiling doesn't add too much overhead</p>



<a name="157277198"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157277198" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157277198">(Jan 31 2019 at 14:57)</a>:</h4>
<p>Would plain text files in a folder structure be ok? Something like <code>perf_data/{category_name}/{query_name}/stack_traces/{timestamp}.txt</code></p>



<a name="157277303"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157277303" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157277303">(Jan 31 2019 at 14:58)</a>:</h4>
<p>I guess at that point there could be a <code>perf_data/log.txt</code> at the root which has the raw data in it with timestamps like <span class="user-mention" data-user-id="124287">@mw</span> wanted</p>



<a name="157277499"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157277499" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157277499">(Jan 31 2019 at 15:01)</a>:</h4>
<p>I think that there aren't many restrictions on how the data is written to disk because there'll be a post-processing step outside the compiler anyway</p>



<a name="157277560"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157277560" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157277560">(Jan 31 2019 at 15:01)</a>:</h4>
<p><span class="user-mention" data-user-id="125250">@Wesley Wiser</span> I would prefer one big file I think</p>



<a name="157277569"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157277569" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157277569">(Jan 31 2019 at 15:01)</a>:</h4>
<p>but I guess whatever is fine</p>



<a name="157277637"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157277637" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157277637">(Jan 31 2019 at 15:02)</a>:</h4>
<p>maybe it's better to have a directory; seems a bit harder to load up</p>



<a name="157277732"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157277732" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157277732">(Jan 31 2019 at 15:03)</a>:</h4>
<p>I don't feel strongly either way. Just want to get it to be whatever you find useful :)</p>



<a name="157277846"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157277846" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157277846">(Jan 31 2019 at 15:05)</a>:</h4>
<p>maybe allowing for one file per thread would make the implementation simpler</p>



<a name="157277857"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157277857" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157277857">(Jan 31 2019 at 15:05)</a>:</h4>
<p>yeah I mean ultimately anything is fine</p>



<a name="157277861"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157277861" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157277861">(Jan 31 2019 at 15:05)</a>:</h4>
<p>so i'd sort of be guided by the impl</p>



<a name="157277873"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157277873" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157277873">(Jan 31 2019 at 15:05)</a>:</h4>
<p>I'm semi-inclined not to do a JSON-representation</p>



<a name="157277928"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157277928" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157277928">(Jan 31 2019 at 15:06)</a>:</h4>
<p>because that will require serde etc etc, and I found with perf-focus that time to construct the final data structure is kind of important</p>



<a name="157277944"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157277944" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157277944">(Jan 31 2019 at 15:06)</a>:</h4>
<p>so something that let me just charge through line-by-line and push onto some vectors was good</p>



<a name="157277968"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157277968" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157277968">(Jan 31 2019 at 15:06)</a>:</h4>
<p>but then against JSON is kind of nice :P and serde is of course fast, it's more that then you have this intermediate thing you have to convert from. There are probably ways to do this in a streaming fashion with serde, I don't really know</p>



<a name="157277977"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157277977" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157277977">(Jan 31 2019 at 15:06)</a>:</h4>
<p>yeah, having some like JSON for something simple like this is probably overkill</p>



<a name="157278096"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157278096" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157278096">(Jan 31 2019 at 15:08)</a>:</h4>
<p>Ok. I'll try the "one large file" approach for now and we can see how that feels</p>



<a name="157278155"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157278155" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157278155">(Jan 31 2019 at 15:08)</a>:</h4>
<p>I'm trying to a PR with these changes up for review before I leave on Saturday for All Hands</p>



<a name="157278212"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157278212" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157278212">(Jan 31 2019 at 15:09)</a>:</h4>
<p>(random question, and definitely <em>not</em> for the first pass: I wonder if we can easily emit a gzip'd version in the end, and then easily gunzip when loading; I imagine there will be a lot of repetition. Presumably we have some rust bindings to some such compression library.)</p>



<a name="157278329"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157278329" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157278329">(Jan 31 2019 at 15:10)</a>:</h4>
<p>is that something the compiler would need to do?</p>



<a name="157278393"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157278393" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157278393">(Jan 31 2019 at 15:11)</a>:</h4>
<p>Seems possible <a href="https://crates.io/crates/flate2" target="_blank" title="https://crates.io/crates/flate2">https://crates.io/crates/flate2</a></p>



<a name="157278418"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157278418" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> QuietMisdreavus <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157278418">(Jan 31 2019 at 15:11)</a>:</h4>
<p>worth noting, i dunno how many files you were thinking of creating if you didn't go with the "one huge file" approach, but emitting loads of files is itself the biggest performance bottleneck in rustdoc</p>



<a name="157278487"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157278487" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> QuietMisdreavus <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157278487">(Jan 31 2019 at 15:12)</a>:</h4>
<p>so it may slow down the collection process to have a lot of little files</p>



<a name="157278583"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157278583" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157278583">(Jan 31 2019 at 15:13)</a>:</h4>
<blockquote>
<p>is that something the compiler would need to do?</p>
</blockquote>
<p>not necessarily</p>



<a name="157278648"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/-Zself-profile/near/157278648" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/-Zself-profile.html#157278648">(Jan 31 2019 at 15:14)</a>:</h4>
<p><span class="user-mention" data-user-id="133692">@QuietMisdreavus</span> That's a good point. Right now this all runs at the very end of compilation so there shouldn't be IO performance issues during collection. Perhaps it won't be feasible to continue capturing everything into memory directly while getting stack traces though...</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>